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ABSTRACT 



This paper presents an Evaluation Scale which is applicable 
to various types of educational Web sites. It consists of three parts: 
composition of the site (concerned with general characteristics, presentation 
and illustration of information, and user*s impression); pedagogical aspects; 
and curriculum-related aspects. Twenty-seven biology teachers who had 
previously passed an advanced in-service course in computer usage in their 
subject formed an expert group. They evaluated two Web sites, ranking 43 
different statements in the Evaluation Scale according to their opinions of 
the presented sites. Evaluation results are presented, along with six figures 
showing the front pages of the two Web sites and response tabulations. The 
paper concludes that the Evaluation Scale should be applicable to 
instructional Web sites of different countries. (AEF) 
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Abstract: As a result of the present work the criteria for evaluating educational Web sites 
have been worked out. A special Evaluation Scale considering curriculum guidelines have 
been developed. The presented Scale is supposed to be ^plicable in evaluation of different 
types of educational Web sites. It consists of three parts: composition of the site, pedagogical 
aspects and curriculum -related aspects. The first part - composition of the site - is divided into 
three units: general characteristics, presentation and illustration of information and user's im- 
pression. 27 teachers who had previously passed an advanced in-service course in usage of 
computers in school biology formed an expert group. They examined and evaluated two edu- 
cational Web sites - "Estonian Vertebrates" and "Arthropods". Teachers ranked 43 different 
statements in the Evaluation Scale according to their opinions of the presented Web sites. The 
analysis of their answers demonstrated the advantage of "Estonian Vertebrates" in some as- 
pects compared with "Arthropods". 



Introduction 

There are numbers of educational recourses available on Internet varying in their composition and contents. 
Some of these have been assigned to educators, the others directly to basic or secondary school students. Several 
textbooks provide information about the composition and design of effective instructional Web site (e.g. Ed- 
wards & Holland, 1994; Boyle, 1997); some of these include curriculum context (e.g. Jones & Scrimshaw, 1988; 
Ross & Scanlon, 1995). The efficiency of application of educational Web sites in learning process depends on 
many aspects. Therefore, it is essential to clarify all the evaluation aspects related to the educational Internet re- 
sources. Obviously the same criteria can be considered both in creating effective instructional Web site and in 
the evaluation of educational resources available on Internet. As a minimum, the evaluation process must take 
into consideration technical and pedagogical aspects. A good example connecting two mentioned aspects can be 
examined on Internet (Jones, 1998). However, the evaluation of educational Web pages must also include the 
curriculum- and subject-related aspects. 
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We selected three different groups of questions essential for evaluating an effective educational Web site: 
composition of interface, general pedagogical considerations and applicability in curriculum context. According 
to this the proposed Evaluation Scale of educational Web Site consisted of three parts: composition of the site, 
pedagogical aspects and curriculum-related aspects. Each question was presented as a statement. Teachers had to 
evaluate all the statements using the five-point scale, where the value of 1 corresponded to complete disagree- 
ment and 5 - to complete agreement. They could choose 0 when they could not evaluate the statement or find the 
answer. 

The first part of the Evaluation Scale - composition of the site - is concerned with general characteristics, 
presentation and illustration of information and user's impression. This part can be titled as the Web-based De- 
sign Scale and was adopted from Jones (1998). 

The first unit of this part embraces general characteristics. The target audience and the objectives should al- 
ways be taken into consideration in composing educational Web site. Everybody starting to examine the educa- 
tional Web site must get a quick survey of the orientation and the scope of the site. Similar design principles, 
suitable navigation devices and appropriate layout enhance finding information and thereby facilitate the learn- 
ing process. Obviously there are some differences in the application of instructional Web site between the stu- 
dents of basic and secondary school. 
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I. Composition of the Site 

A. General Characteristics 



1. It is easy to open the program. 

2. The objective of the site is clearly presented. 

3. The target audience of the presented material is clearly comprehensible. 

4. It is easy to estimate the scope of the presented information. 

5. The site is a complete unit. 

6. The layout of all the presented material is appropriate. 

7. The same design principles are used. 

8. The navigation in the site is easy. 

9. The navigation devices are suitable for basic school students. 

10. The navigation devices are suitable for secondary school students. 



( 1-2-3-4-5 0 ) 
( 1-2-3-4-5 0 ) 
( 1-2-3-4-5 0 ) 
( 1-2-3-4-5 0 ) 
( 1-2-3-4-5 0 ) 
( 1-2-3-4-5 0 ) 
( 1-2-3-4-5 0 ) 
( 1-2-3-4-5 0 ) 
( 1-2-3-4-5 0 ) 
( 1-2-3-4-5 0 ) 



The second unit of the first part concentrates on the presentation and illustration of information. Design of 
the front page must be attractive and all the information on it clearly understandable for the learners. The appro- 
priate structure of hypertext, illustrations, sound, video and animation must form a complete unit. All these as- 
pects enhance the presentation of information and overall the effectiveness of the application of educational Web 
site. 



B. Presentation and Illustration of Information 



1 1 . The front page is reasonable and comprehensible. 

12. The layout of the text supports reading. 

13. The appropriate font is used. 

14. The colors of the text are reasonable and consistent. 

15. The quality of illustrations is good. 

16. The photos enhance the presentation of the information. 

17. The figures enhance the presentation of the information. 

18. The audio enhances the presentation of the information. 

19. The video enhances the presentation of the information. 

20. The animation enhances the presentation of the information. 



( 1-2-3-4-5 0 ) 
( 1-2-3-4-5 0 ) 
( 1-2-3-4-5 0 ) 
( 1-2-3-4-5 0 ) 
( 1-2-3-4-5 0 ) 
( 1-2-3-4-5 0 ) 
( 1-2-3-4-5 0 ) 
( 1-2-3-4-5 0 ) 
( 1-2-3-4-5 0 ) 
( 1-2-3-4-5 0 ) 



The user’s positive impressions increase the learner’s interest to the subject and therefore increase the learning 
motivation. According to this there were some questions about user's impressions in the third unit of the first 
part. However, the opinions of evaluators might be different - teachers and students are not always of one mind. 



C. User’s Impression 

( 1-2-3-4-5 0 ) 
( 1-2-3-4-5 0 ) 
( 1-2-3-4-5 0 ) 
( 1-2-3-4-5 0 ) 



21 . The site is engaging and invites to use. 

22. The site is composed creatively. 

23. The site is well done. 

24. The site is enjoyable. 



The second part of the Evaluation Scale rates several attitudes on educational philosophy, goal orientation, 
motivation, metacognitive support, and advantage of learning strategies and structural flexibility. All these as- 
pects must be considered in designing an effective educational Web site (Reeves & Reeves, 1997; Philips, 1997). 
The evaluation of pedagogical aspects presumes the appropriate educational background. Therefore, teachers 
were provided with some additional explanations in our case study. 

II. Pedagogical Aspects 

25. Pedagogical philosophy (instructivist - constructivist). ( I-2-3-4-5 0 ) 

26. Goal orientation (sharply focused- general). ( 1-2-3-4-5 0 ) 

27. Source of motivation (extrinsic - intrinsic). ( 1-2-3-4-5 0 ) 

28. Teacher’s presence in using of the site (not necessary - obligatory). ( 1-2-3-4-5 0 ) 

29. Metacognitive support (does not provide feedback - provides feedback). ( I-2-3-4-5 0 ) 
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30. Collaborative learning strategies (unsupported - supported). 

31 . Structural flexibility (fixed - open). 



( 1-2-3-4-5 0 ) 
( 1-2-3-4-5 0 ) 



The third part of the Scale embraces curriculum-related aspects. In this part an essential group of questions 
includes the relation of educational Web site with the concepts of Estonian new State Curriculum. However, the 
same guidelines can be found in the curricula of the most countries. We were interested if the presented Web site 
is applicable in teaching various themes of curriculum, but also in extra-curriculum activities. One obvious pre- 
requisite for successful usage of instructional Web site is that all the texts and presented materials are correct in 
its content and comprehensible by the users. The application of some instructional Web sites might be accompa- 
nied with supplementary materials. Therefore, this aspect was also under consideration. 



111. Curriculum-Related Aspects 

32. The site is related to the themes of the curriculum. 

33. The site is applicable while teaching various themes of curriculum. 

34. The site is also applicable in several extra-curriculum activities. 

35. The presented material is correct in its content. 

36. The content of the information is comprehensible for basic school students. 

37. The content of the information is comprehensible for secondary school students. 

38. The formulation of texts is well adapted to basic school students. 

39. The formulation of texts is well adapted to secondary school students. 

40. There are lots of incomprehensible terms in the topics. 

41 . All the terms are well explained. 

42. The material of the site is applicable without supplementary materials. 

43. The material of the site is usable in computerized biology classes. 
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Educational Web Sites 

Two educational Web sites have been examined in the evaluation process - ’’Estonian Vertebrates” and ’’Ar- 
thropods” (Fig. 1). The first one has specially been composed for basic and secondary school students in our De- 
partment. It considers the guidelines of Estonian new Sate Curriculum (Sarapuu, 1997). The second one - ’’Ar- 
thropods” - has been designed in the Institute of Zoology and Botany (Estonia). In developing the latter no spe- 
cial efforts towards the curriculum content were done. 

The main part of ’’Estonian Vertebrates” Web site {http://sunsite.ee/loomad/) is a database about five groups 
of Estonian vertebrates: fishes, amphibians, reptiles, birds and mammals. It includes general information about 
vertebrates, lists of all the species of Estonian vertebrates and survey of all the groups emphasizing the local Es- 
tonian aspects. The most common (ca 300) species are provided with detailed descriptions, color photos and 
voices. ’’Estonian Vertebrates” has been developed in order to be applicable in biology classes both in basic and 
secondary school. Therefore the information about each species has been presented on two Web pages consid- 
ering students’ different age and abilities. The first page gives general information about particular species in- 
cluding its appearance, distribution, abundance, feeding, reproduction, development and endangerment. This one 
is mainly orientated to the basic school students. Considering this, the first page has fewer scientific facts and 
simpler vocabulary whereas the second one - supplementary information page - has some more terms and scien- 
tific information. This additional material is mainly assigned to the secondary school students (Adojaan & 
Sarapuu, 1997). 

The principles of composition of ’’Arthropods” Web site {http://www.zbi.ee/satikad/) are completely differ- 
ent. It gives a systematic survey of all the classes and orders of Arthropods. It is composed as an attractive sup- 
plementary reading material to encourage students to study Estonian nature. Different groups of Arthropods are 
characterized by their morphology, anatomy and physiology (e.g. reproduction, development, feeding, adapta- 
tions, etc.). An essential part of each topic is the information about the behavior and adaptation of Arthropods. 
The data of their ecology are concerned with interactions with environment, plants and other animals. ’’Arthro- 
pods” Web pages are illustrated with original colour photos, schemes and drawings. The presented material is 
not so directly meant for the basic or secondary school students, but for everybody interested in the life of Ar- 
thropods and generally in Estonian nature. 




4 





Fig. 1. The front pages of "Estonian Vertebrates" (on the left) and "Arthropods" (on the right) Web sites. 



An Expert Group 

An expert group consisting of 27 biology teachers had previously passed an advanced in-service course in 
usage of computers in school biology. Teachers came from different regions of Estonia and had some previous 
experience in the application of computers in teaching biology in basic or secondary school. 

At first, an introductory lecture about the concepts of evaluation of educational software and instructional 
Web sites was conducted to the teachers. Subsequently, they had the opportunity to get acquainted with two Web 
sites - "Estonian Vertebrates" and "Arthropods". At the same time they participated in the evaluation process 
based on the Scale described in the previous section. 



Results 



27 biology teachers evaluated "Estonian Vertebrates" Web site and 22 of those ranked the proposed aspects 
of "Arthropods". The first unit of the first part of the Evaluation Scale was concerned with general characteris- 
tics. The average ranks calculated from the teachers' opinions (Fig. 2) revealed that the target audience and the 
objective of "Estonian Vertebrates" Web site are more clearly presented (questions 2 and 3, respectively). This 
result is closely related to the question number 1 1 of the second unit - "the front page is reasonable and compre- 
hensible" (Fig. 3). The front page of "Estonian Vertebrates" is more informative, than the first page of "Arthro- 
pods" which appears to be more illustrative than informative. Teachers also thought that the navigation devices 
on "Estonian Vertebrates" Web site were more convenient (question 8). 






el 





Question 



Fig. 2. The average ranks of the general characteristics of "Estonian Vertebrates" (+) and "Arthropods" 
sites. 
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The second unit was concerned with the presentation and illustration of information (Fig. 3). One of the 
greatest advantages of "Estonian Vertebrates" Web site is that it is provided with real voices of nature - several 
species are characterized not only with text but also with their voices (question 18). As far as birds are con- 
cerned, their song is an essential feature in recognizing them in nature. Neither "Estonian Vertebrates" nor "Ar- 
thropods" Web site includes video and animation. Therefore, some questions of this unit are not applicable with 
these Web sites. 




Fig. 3. The average ranks of the presentation and illustration of information on "Estonian Vertebrates" (+) and 
"Arthropods" ( ) Web sites. 

The third unit asked for the user's impressions. Teachers valued "Estonian Vertebrates" Web site higher in all 
the aspects (Fig. 4). This result proves that "Estonian Vertebrates" Web pages are more attractive and enjoyable. 
Considering this it is suggested that the material presented on "Estonian Vertebrates" Web site enhances the 
learner's motivation and interest in school biology. 
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Fig. 4. The average ranks of the user's impression concerning "Estonian Vertebrates" (+) and "Arthropods" ( ) 
Web sites. 

The second part of the present Evaluation Scale studied pedagogical aspects of two Web sites. There were no 
essential differences in the teachers' opinions (Fig. 5). The "Estonian Vertebrates" was thought to be more gener- 



ally focused compared with the orientation of "Arthropods" Web site (question 26). The teachers' answers also 
revealed that "Arthropods" provides more possibilities for feedback (question 29). 




Question 



Fig. 5. The average ranks of the pedagogical aspects on "Estonian Vertebrates" (+) and "Arthropods" ( ) Web 
sites. 

The last part of the Evaluation Scale rated the curriculum-related aspects (Fig. 6). Both sites are applicable in 
teaching various themes of curriculum (questions 32, 33), but also in extra-curriculum activities (34). The main 
difference between "Estonian Vertebrates" and "Arthropods" Web sites is in their vocabulary. There are more in- 
comprehensible terms in the topics of "Arthropods" (40). The scientific terms are well explained in the texts of 
"Estonian Vertebrates" (41) and therefore the presented material is better comprehensible by the basic school 
students (36). The successful application of "Arthropods" Web site depends on supplementary materials (42). 




Question 



Fig. 6. The average ranks of the curriculum-related aspects on "Estonian Vertebrates" (+) and "Arthropods" ( ) 
Web sites. 



Several criteria for evaluating the educational Web sites have been chosen as a result of the present work. 
Consequently a special Evaluation Scale consisting of 43 questions has been worked out. The Evaluation Scale 



considers several curriculum-related aspects. This is the main difference of the proposed scale compared with the 
others. It is supposed that present Evaluation Scale is not only applicable with educational Web sites connected 
with Estonian State Curriculum but also with various instructional Web sites of different countries. 
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